Method and system for generating a high resolution image

ABSTRACT

A method for generating an image is provided. The method includes estimating a high resolution image from a plurality of low resolution images and downsampling the estimated high resolution image to obtain estimates of a plurality of low resolution images. The method also includes generating a desired high resolution image based upon comparison of the downsampled low resolution images and the plurality of low resolution images.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a Continuation of U.S. application Ser. No.12/627,725, filed on Nov. 30, 2009, the entirety of which isincorporated herein by reference.

BACKGROUND

High resolution images are desirable for a variety of applications suchas in the areas of medical imaging, remote sensing, forensic imaging,and robotics applications, among others. Typically, high resolutionimages are obtained by combining information from a set of lowresolution images obtained from two-dimensional scenes from differentviewpoints. It also involves estimation of parametric relative motion.However, this technique may not be valid for three-dimensional scenes,in which case the motion estimation requires stereo disparityestimation.

Traditional stereo algorithms estimate disparity at the same resolutionas that of the low resolution observed images which are assumed asaveraged versions of a corresponding high resolution image. However,pixel averaging at low resolution may cause distortions in the estimateddisparity. As a result, shape details may not be preserved in the highresolution depth map/stereo disparity map of the obtained image.

SUMMARY

The foregoing summary is illustrative only and is not intended to be inany way limiting. In addition to the illustrative aspects, embodiments,and features described above, further aspects, embodiments, and featureswill become apparent by reference to the drawings and the followingdetailed description.

Briefly, in accordance with one aspect, a method for generating an imageis provided. The method includes estimating a high resolution image froma plurality of low resolution images and downsampling the estimated highresolution image to obtain estimates of a plurality of low resolutionimages. The method also includes generating a desired high resolutionimage based upon comparison of the downsampled low resolution images andthe plurality of low resolution images.

In accordance with another aspect, a method for generating an image isprovided. The method includes (a) estimating a high resolution imagefrom a plurality of low resolution images, (b) downsampling a pluralityof low resolution images from the high resolution image and (c)determining a penalty function value based upon the downsampled lowresolution images and the plurality of images. The method also includes(d) correcting the high resolution image by reference to a depthmap/stereo disparity map, (e) downsampling the corrected low resolutionimages from the corrected high resolution image and repeating steps (c),(d) and (e) until a desired high resolution image is obtained.

In accordance with another aspect, an image generation system isprovided. The image generation system includes a memory configured tostore a plurality of low resolution images of a single subject fromdifferent spatial viewpoints and an image processing circuit configuredto estimate a high resolution image from the plurality of low resolutionimages and to generate a desired high resolution image based uponcomparison of the estimated high resolution image, the plurality ofimages, and pixel shifts representative of a depth of pixels anddifferent spatial viewpoints.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is an example flow diagram of an embodiment of a method forgenerating a high resolution image.

FIG. 2 is an illustrative embodiment of an image generation system.

FIG. 3 illustrates example images generated by conventional imageprocessing methods and by using the image generation system of FIG. 2.

FIG. 4 is a block diagram illustrating an example computing device thatis arranged for generating a high resolution image.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying drawings, which form a part hereof. In the drawings,similar symbols typically identify similar components, unless contextdictates otherwise. The illustrative embodiments described in thedetailed description, drawings, and claims are not meant to be limiting.Other embodiments may be utilized, and other changes may be made,without departing from the spirit or scope of the subject matterpresented herein. It will be readily understood that the aspects of thepresent disclosure, as generally described herein, and illustrated inthe Figures, can be arranged, substituted, combined, separated, anddesigned in a wide variety of different configurations, all of which areexplicitly contemplated herein.

Example embodiments are generally directed to enhancement of imageresolution. Such techniques may be useful in obtaining images with highimage resolution that facilitate analysis, diagnosis or other purposesin a variety of applications.

Referring now to FIG. 1, an example flow diagram 100 of an embodiment ofa method for generating a high resolution image is illustrated. As usedherein, the term “high resolution image” refers to a high resolutionintensity image and high resolution depth map/stereo disparity map. Atblock 110, a plurality of low resolution images are obtained. In thisexample embodiment, the plurality of low resolution images comprisestereo images of a single subject/scene from different spatial viewpoints. In particular, the plurality of low resolution images arecaptured as spatially shifted versions of the same subject/scene, withthe shifts occurring at the sub-pixel levels. In one example embodiment,the plurality of low resolution images are captured as time-shiftedversions of the same subject/scene.

In one embodiment, the stereo images are obtained from a plurality ofstereo cameras disposed at different locations. In another embodiment,the stereo images are obtained by translation of a single stereo camerain different spatial viewpoints. In another example embodiment, thestereo images include images of a moving object captured using a camera.It should be noted that the stereo images may be obtained from knowndifferent spatial viewpoints and the resulting high resolution image maycorrespond to one of the known spatial view points in the input set, ormay correspond to a different view.

At block 120, a high resolution image is estimated from the plurality oflow resolution images. In this embodiment, the high resolution image isestimated by interpolation of the plurality of low resolution images.For example, the high resolution image may be estimated using bilinearinterpolation of the low resolution images. In another exampleembodiment, the high resolution image may be estimated using bicubicinterpolation of the low resolution images. Further, the estimated highresolution image is downsampled to obtain estimates of low resolutionimages (block 130).

In one example, the estimates of downsampled low resolution images aregenerated by averaging of pixels of the estimated high resolution imagesusing a downsampling operator. However, a variety of other downsamplingtechniques may be employed to obtain low resolution images from theestimated high resolution image. Such estimates of downsampled lowresolution images are then utilized to generate a desired highresolution image based upon comparison of these downsampled lowresolution image estimates and the plurality of low resolution images,as described below.

At block 140, a penalty function value is determined based upon thedownsampled low resolution images and the plurality of low resolutionimages. Moreover, the penalty function value is progressively reduced togenerate a corrected high resolution image by iteratively estimatingpixel values for the desired high resolution image based upon theestimated high resolution image and a depth map/stereo disparity map(block 150).

At block 160, corrected low resolution images are obtained bydownsampling the corrected high resolution image. Further, it isverified if the corrected high resolution image has a desired resolutionby comparing the penalty function value with a pre-determined threshold(block 170). If the corrected high resolution image is of the desiredresolution then the image is stored in a memory or is displayed to auser (block 180). If a desired resolution of the high resolution imageis not obtained, then the penalty function value is re-estimated usingthe corrected low resolution images (block 140) and the penalty functionvalue is further minimized until the desired high resolution image isobtained.

FIG. 2 illustrates an example image generation system 200. The imagegeneration system 200 includes a memory 210 configured to store aplurality of low resolution images such as represented by referencenumerals 220 and 230 of a single subject 240 obtained from differentspatial viewpoints. In this example embodiment, the low resolutionimages 220 and 230 are obtained using stereo cameras 250 and 260respectively. In another embodiment, the low resolution images 220 and230 may be obtained by translation of a single camera. In certainembodiments, the image generation system 200 includes a monochromecamera or a color camera, or a conventional film. In certain otherembodiments, the image generation system 200 includes an infraredcamera, or a thermal camera.

In certain example embodiments, the system 200 may include anillumination device (not shown) for illuminating the subject 240 and animage capture device (such as the camera 250) configured to capture thelow resolution images 220 and 230 of the subject 240. Image capture maybe performed by using any suitable illumination device and other imagingoptics arrangement with possible configurations ranging from a singlelens component to a multi-element lens.

It should be borne in mind that, although a single memory is describedhere, the storing function may be performed by more than one memorydevices associated with the system for storing image processingroutines, low resolution images, and so forth.

The memory 210 may include hard disk drives, optical drives, tapedrives, random access memory (RAM), read-only memory (ROM), programmableread-only memory (PROM), redundant arrays of independent disks (RAID),flash memory, magneto-optical memory, holographic memory, bubble memory,magnetic drum, memory stick, Mylar® tape, smartdisk, thin film memory,zip drive, and so forth.

Referring again to FIG. 2, the image generation system 200 also includesan image processing circuit 270 configured to estimate a high resolutionimage 280 from low resolution images 220 and 230 using the technique asdescribed above with reference to FIG. 1. It should be noted that thelow resolution images 220 and 230 of the subject 240 may bepre-generated and stored in the memory 210. Further, such images 220 and230 may be accessed by the image processing circuit 270 to generate adesired high resolution image of the subject 240.

The high resolution image may be estimated by interpolation of the lowresolution images 220 and 230. The image processing circuit 270 isconfigured to generate a desired high resolution image based uponcomparison of the estimated high resolution image, the plurality of lowresolution images 220 and 230 and the high resolution depth/stereodisparity.

In this embodiment, the estimated high resolution image is downsampledto obtain downsampled low resolution images. In certain embodiments, theresolution of the downsampled low resolution images is substantiallyequal to the resolution of the plurality of low resolution images, suchas represented by reference numerals 220 and 230 obtained by the cameras250 and 260 respectively. In one example embodiment, the estimated highresolution image is downsampled to obtain two downsampled low resolutionimages. The low resolution images (y_(i)) of size N₁×N₂ are downsampledfrom the estimated high resolution image (x) of size L₁×L₂ and arerepresented by the following relationship:

y _(i) =DW _(i) x+η _(i)  (1)

Where:

-   -   y_(i) is the i^(th) low resolution image;    -   D is the downsampling matrix;    -   W_(i) is the warping matrix; and    -   η_(i) is a noise component.

In this example embodiment, the downsampling matrix D uniformly averagesL₁/N₁×L₂/N₂ high resolution pixels to one low resolution pixel. In thisexample embodiment, low resolution images downsampled from the estimatedhigh resolution images may be represented by the following equations:

$\begin{matrix}{{y_{1}( {n_{1},n_{2}} )} = {{\sum\limits_{l_{1},{l_{2} \in G}}{{d( {n_{1},n_{2},l_{1},l_{2}} )} \cdot {x( {l_{1},l_{2}} )}}} + {\eta_{1}( {n_{1},n_{2}} )}}} & (2) \\{{y_{2}( {n_{1},n_{2}} )} = {{\sum\limits_{l_{1},{l_{2} \in G}}{{d( {n_{1},n_{2},l_{1},l_{2}} )} \cdot {x( {{l_{1} - {\delta ( l_{1} )}},l_{2}} )}}} + {\eta_{2}( {n_{1},n_{2}} )}}} & (3)\end{matrix}$

Where: δ(l₁) is the disparity for the (l₁, l₂)^(th) pixel;

-   -   G is the group of high resolution pixels that average to a low        resolution pixel;        -   d(n₁,n₂,l₁,l₂) is the down-sampling operator that averages            the pixels in the high resolution group G resulting in the            low resolution pixel (n₁,n₂); and    -   η₁(n₁,n₂) and η₂(n₁,n₂) are noise components.

Furthermore, the desired high resolution image 280 is generated byprogressively reducing a penalty function value based upon thedownsampled low resolution images and the plurality of low resolutionimages 220 and 230. In this example embodiment, the penalty functionvalue is estimated in accordance with the following relationship:

E=E _(d)(δ,x)+E _(δ)(δ)+E _(x)(x)  (4)

Where:

-   -   E_(d)(δ,x) is the data term of the penalty function value;    -   E_(δ)(δ) is a prior applied to the disparity; and    -   E_(x)(x) is a prior applied to the image.

In the illustrated embodiment, the priors E_(δ)(δ) and E_(x)(x) includeMarkov Random Fields (MRF) priors applied to the disparity and the imagerespectively which facilitate reduction of noise in the high resolutionimage 280. In certain embodiments, occlusions due to undesirable objectsare substantially reduced in the high resolution image 280. Inparticular, occluded pixels are detected in the estimated highresolution image 280 and the penalty function value for non-occludedpixels in the estimated high resolution image 280 is reduced to obtainthe desired high resolution image.

In this example embodiment, the data term E_(d)(δ,x) of the penaltyfunction value is estimated in accordance with the followingrelationship:

$\begin{matrix}{{E_{d}( {\delta,x} )} = {{\sum\limits_{n_{1},{n_{2} = 1}}^{N_{1},N_{2}}( {{y_{1}( {n_{1},n_{2}} )} - {\sum\limits_{l_{1},{l_{2} \in G}}{{d( {n_{1},n_{2},l_{1},l_{2}} )}{x( {l_{1},l_{2}} )}}}} )^{2}} + {\sum\limits_{n_{1},{n_{2} \in O^{\prime}}}( {{y_{2}( {n_{1},n_{2}} )} - {\sum\limits_{l_{1},{l_{2} \in G}}{{d( {n_{1},n_{2},l_{1},l_{2}} )}{x( {{\theta ( l_{1} )},l_{2}} )}}}} )^{2}} + {\sum\limits_{n_{1},{n_{2} \in O}}\lambda_{OCC}}}} & (5)\end{matrix}$

Where:

-   -   θ(l₁)=l₁−δ(l₁);    -   O is the set of low resolution pixels each of which involves        contribution from a high resolution site for which the high        resolution pixel is occluded; and    -   O′ is the compliment of O.

Further, the priors E_(δ)(δ) and E_(x)(x) applied to the disparity andthe image respectively are represented by the following relationships:

$\begin{matrix}{{E_{\delta}(\delta)} = {\sum\limits_{c \in C_{\delta}}{v_{c}^{\delta}(\delta)}}} & (6) \\{{E_{x}(x)} = {\sum\limits_{c \in C_{x}}{v_{c}^{x}(x)}}} & (7)\end{matrix}$

Where:

-   -   c denotes a clique; and    -   C_(δ) and C_(x) represent set of all cliques for the high        resolution disparity and image.

In this example embodiment, the penalty function value is alternativelyminimized for the image and the depth map/stereo disparity map usingiterated conditional modes (ICM) and alpha-expansion graph cuttechniques respectively. However, a variety of other suitable techniquesmay be employed for minimization of the penalty function value. Thus, inalternate iterations, the current estimate of one of the image and depthmap/stereo disparity map is utilized to determine the other variable.The penalty function value is progressively reduced until the value isbelow a pre-determined threshold to achieve the desired high resolutionimage 280. In this example embodiment, the threshold is configurable andmay be defined by a user of the system 200. The desired high resolutionimage 280 is sent to an output of the system 200. In this exampleembodiment, the desired high resolution image 280 may be displayed to auser of the system through a display 290.

In certain embodiments, the priors E_(δ)(δ) and E_(x)(x) are selected toobtain a smooth solution for the high resolution image 280. In oneexample embodiment, a truncated absolute function is selected for thedisparity prior E_(δ)(δ) and a discontinuity adaptive MRF prior isselected for the image prior E_(x)(x) In this embodiment, thediscontinuity adaptive prior is represented by the followingrelationship:

$\begin{matrix}{{E_{x}(x)} = {\gamma - {\gamma \; e\; \frac{- ( {{x( {t,j} )} - {x( {p,q} )}} )^{2}}{\gamma}}}} & (8)\end{matrix}$

Where:

-   -   x(i,j) and x(p,q) are neighboring pixels in the high resolution        image; and    -   γ is a parameter that controls the discontinuity adaptiveness of        the prior.    -   Advantageously, the priors described above facilitate robustness        of the image estimation to any errors in disparity in addition        to preserving any discontinuities and other details of the        image.

FIG. 3 illustrates example images 300 generated by conventional imageprocessing methods and by using the system 200 of FIG. 2. In thisexample embodiment, image generated with interpolated disparity computedby conventional stereo on low resolution images is represented byreference numeral 310. Further, image generated with high resolutiondisparity computed using stereo on interpolated images is represented byreference numeral 320.

Further, an image obtained using the technique described above andreference ground truth image are represented by reference numerals 330and 340 respectively. As can be seen, the image 330 has lesser artifactsand distortions in the background as compared to images 310 and 320 andpreserves the shapes of the objects being imaged. Moreover, the outputimage 330 is closer to the reference ground truth image 340 as comparedto the images 310 and 320.

In this example, a bicubic interpolated image and a high resolutionthree-dimensional image obtained using the image generation system 200of FIG. 2 are represented by reference numerals 350 and 360respectively. As can be seen, the image 360 has substantially highresolution as compared to the bicubic interpolated image 350 andpreserves the details of the imaged scene.

The example methods and systems described above facilitate generation ofhigh resolution images from low resolution images provide resolutionenhancement of images using non-parametric depth dependent pixel motion.

The image generation technique described above may be utilized in avariety of applications such as for generating three-dimensional modelsfrom low resolution images obtained from cell phones, for example. Thetechnique may be utilized for rendering high resolution models forapplications that need preservation of image details, such as heritagesites. Further, the technique also facilitates generation of images withincomplete data due to occlusions and so forth.

As will be appreciated by those of ordinary skill in the art, theforegoing example, demonstrations, and process steps may be implementedby suitable code on a processor-based system. It should also be notedthat different implementations of the present technique may perform someor all of the steps described herein in different orders orsubstantially concurrently, that is, in parallel.

Furthermore, the functions may be implemented in a variety ofprogramming languages, such as C++ or JAVA. Such code, as will beappreciated by those of ordinary skill in the art, may be stored oradapted for storage on one or more tangible, machine readable media,such as on memory chips, local or remote hard disks, optical disks (thatis, CDs or DVDs), or other media, which may be accessed by aprocessor-based system to execute the stored code.

FIG. 4 is a block diagram illustrating an example computing device 400that is arranged for generating high resolution images from lowresolution images in accordance with the present disclosure. In a verybasic configuration 402, computing device 400 typically includes one ormore processors 404 and a system memory 406. A memory bus 408 may beused for communicating between processor 404 and system memory 406.

Depending on the desired configuration, processor 404 may be of any typeincluding but not limited to a microprocessor (μP), a microcontroller(μC), a digital signal processor (DSP), or any combination thereof.Processor 404 may include one more levels of caching, such as a levelone cache 410 and a level two cache 412, a processor core 414, andregisters 416. An example processor core 414 may include an arithmeticlogic unit (ALU), a floating point unit (FPU), a digital signalprocessing core (DSP Core), or any combination thereof. An examplememory controller 418 may also be used with processor 404, or in someimplementations memory controller 418 may be an internal part ofprocessor 404.

Depending on the desired configuration, system memory 406 may be of anytype including but not limited to volatile memory (such as RAM),non-volatile memory (such as ROM, flash memory, etc.) or any combinationthereof. System memory 406 may include an operating system 420, one ormore applications 422, and program data 424. Application 422 may includean image processing algorithm 426 that is arranged to perform thefunctions as described herein including those described with respect toprocess 100 of FIG. 1. Program data 424 may include low resolutionimages 428 that may be useful for generating the desired high resolutionimage as is described herein.

In some embodiments, application 422 may be arranged to operate withprogram data 424 on operating system 420 such that generation of thedesired high resolution image based upon comparison of downsampled lowresolution images from an estimated high resolution image and aplurality of obtained low resolution images may be performed. Thisdescribed basic configuration 402 is illustrated in FIG. 4 by thosecomponents within the inner dashed line.

Computing device 400 may have additional features or functionality, andadditional interfaces to facilitate communications between basicconfiguration 402 and any required devices and interfaces. For example,a bus/interface controller 430 may be used to facilitate communicationsbetween basic configuration 402 and one or more data storage devices 432via a storage interface bus 434. Data storage devices 432 may beremovable storage devices 436, non-removable storage devices 438, or acombination thereof.

Examples of removable storage and non-removable storage devices includemagnetic disk devices such as flexible disk drives and hard-disk drives(HDD), optical disk drives such as compact disk (CD) drives or digitalversatile disk (DVD) drives, solid state drives (SSD), and tape drivesto name a few. Example computer storage media may include volatile andnonvolatile, removable and non-removable media implemented in any methodor technology for storage of information, such as computer readableinstructions, data structures, program modules, or other data.

System memory 406, removable storage devices 436 and non-removablestorage devices 438 are examples of computer storage media. Computerstorage media includes, but is not limited to, RAM, ROM, EEPROM, flashmemory or other memory technology, CD-ROM, digital versatile disks (DVD)or other optical storage, magnetic cassettes, magnetic tape, magneticdisk storage or other magnetic storage devices, or any other mediumwhich may be used to store the desired information and which may beaccessed by computing device 400. Any such computer storage media may bepart of computing device 400.

Computing device 400 may also include an interface bus 440 forfacilitating communication from various interface devices (e.g., outputdevices 442, peripheral interfaces 444, and communication devices 446)to basic configuration 402 via bus/interface controller 430. Exampleoutput devices 442 include a graphics processing unit 448 and an audioprocessing unit 450, which may be configured to communicate to variousexternal devices such as a display or speakers via one or more A/V ports452.

Example peripheral interfaces 444 include a serial interface controller454 or a parallel interface controller 456, which may be configured tocommunicate with external devices such as input devices (e.g., keyboard,mouse, pen, voice input device, touch input device, etc.) or otherperipheral devices (e.g., printer, scanner, etc.) via one or more I/Oports 458. An example communication device 646 includes a networkcontroller 460, which may be arranged to facilitate communications withone or more other computing devices 462 over a network communicationlink via one or more communication ports 464.

The network communication link may be one example of a communicationmedia. Communication media may typically be embodied by computerreadable instructions, data structures, program modules, or other datain a modulated data signal, such as a carrier wave or other transportmechanism, and may include any information delivery media. A “modulateddata signal” may be a signal that has one or more of its characteristicsset or changed in such a manner as to encode information in the signal.

By way of example, and not limitation, communication media may includewired media such as a wired network or direct-wired connection, andwireless media such as acoustic, radio frequency (RF), microwave,infrared (IR) and other wireless media. The term computer readable mediaas used herein may include both storage media and communication media.

Computing device 400 may be implemented as a portion of a small-formfactor portable (or mobile) electronic device such as a cell phone, apersonal data assistant (PDA), a personal media player device, awireless web-watch device, a personal headset device, an applicationspecific device, or a hybrid device that include any of the abovefunctions. Computing device 400 may also be implemented as a personalcomputer including both laptop computer and non-laptop computerconfigurations.

The present disclosure is not to be limited in terms of the particularembodiments described in this application, which are intended asillustrations of various aspects. Many modifications and variations canbe made without departing from its spirit and scope, as will be apparentto those skilled in the art. Functionally equivalent methods andapparatuses within the scope of the disclosure, in addition to thoseenumerated herein, will be apparent to those skilled in the art from theforegoing descriptions. Such modifications and variations are intendedto fall within the scope of the appended claims. The present disclosureis to be limited only by the terms of the appended claims, along withthe full scope of equivalents to which such claims are entitled. It isto be understood that this disclosure is not limited to particularmethods, reagents, compounds compositions or biological systems, whichcan, of course, vary. It is also to be understood that the terminologyused herein is for the purpose of describing particular embodimentsonly, and is not intended to be limiting.

With respect to the use of substantially any plural and/or singularterms herein, those having skill in the art can translate from theplural to the singular and/or from the singular to the plural as isappropriate to the context and/or application. The varioussingular/plural permutations may be expressly set forth herein for sakeof clarity.

It will be understood by those within the art that, in general, termsused herein, and especially in the appended claims (e.g., bodies of theappended claims) are generally intended as “open” terms (e.g., the term“including” should be interpreted as “including but not limited to,” theterm “having” should be interpreted as “having at least,” the term“includes” should be interpreted as “includes but is not limited to,”etc.). It will be further understood by those within the art that if aspecific number of an introduced claim recitation is intended, such anintent will be explicitly recited in the claim, and in the absence ofsuch recitation no such intent is present.

For example, as an aid to understanding, the following appended claimsmay contain usage of the introductory phrases “at least one” and “one ormore” to introduce claim recitations. However, the use of such phrasesshould not be construed to imply that the introduction of a claimrecitation by the indefinite articles “a” or “an” limits any particularclaim containing such introduced claim recitation to embodimentscontaining only one such recitation, even when the same claim includesthe introductory phrases “one or more” or “at least one” and indefinitearticles such as “a” or “an” (e.g., “a” and/or “an” should beinterpreted to mean “at least one” or “one or more”); the same holdstrue for the use of definite articles used to introduce claimrecitations.

In addition, even if a specific number of an introduced claim recitationis explicitly recited, those skilled in the art will recognize that suchrecitation should be interpreted to mean at least the recited number(e.g., the bare recitation of “two recitations,” without othermodifiers, means at least two recitations, or two or more recitations).Furthermore, in those instances where a convention analogous to “atleast one of A, B, and C, etc.” is used, in general such a constructionis intended in the sense one having skill in the art would understandthe convention (e.g., “a system having at least one of A, B, and C”would include but not be limited to systems that have A alone, B alone,C alone, A and B together, A and C together, B and C together, and/or A,B, and C together, etc.). In those instances where a conventionanalogous to “at least one of A, B, or C, etc.” is used, in general sucha construction is intended in the sense one having skill in the artwould understand the convention (e.g., “a system having at least one ofA, B, or C” would include but not be limited to systems that have Aalone, B alone, C alone, A and B together, A and C together, B and Ctogether, and/or A, B, and C together, etc.).

It will be further understood by those within the art that virtually anydisjunctive word and/or phrase presenting two or more alternative terms,whether in the description, claims, or drawings, should be understood tocontemplate the possibilities of including one of the terms, either ofthe terms, or both terms. For example, the phrase “A or B” will beunderstood to include the possibilities of “A” or “B” or “A and B.”

As will be understood by one skilled in the art, for any and allpurposes, such as in terms of providing a written description, allranges disclosed herein also encompass any and all possible subrangesand combinations of subranges thereof. Any listed range can be easilyrecognized as sufficiently describing and enabling the same range beingbroken down into at least equal halves, thirds, quarters, fifths,tenths, etc. As a non-limiting example, each range discussed herein canbe readily broken down into a lower third, middle third and upper third,etc.

As will also be understood by one skilled in the art all language suchas “up to,” “at least,” “greater than,” “less than,” and the likeinclude the number recited and refer to ranges which can be subsequentlybroken down into subranges as discussed above. Finally, as will beunderstood by one skilled in the art, a range includes each individualmember. Thus, for example, a group having 1-3 cells refers to groupshaving 1, 2, or 3 cells. Similarly, a group having 1-5 cells refers togroups having 1, 2, 3, 4, or 5 cells, and so forth.

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments will be apparent to those skilled in the art.The various aspects and embodiments disclosed herein are for purposes ofillustration and are not intended to be limiting, with the true scopeand spirit being indicated by the following claims.

We claim:
 1. A method comprising: generating a corrected high resolutionimage by iteratively: identifying a set of occluded pixels in anestimated high resolution image and a set of non-occluded pixels in theestimated high resolution image; determining a value of a penaltyfunction, wherein the penalty function is based on a difference betweena plurality of estimated low resolution images and a plurality of lowresolution images; and adjusting pixel values of the estimated highresolution image to reduce the value of the penalty function for the setof non-occluded pixels.
 2. The method of claim 1, wherein the estimatedhigh resolution image is estimated by interpolation of the plurality oflow resolution images.
 3. The method of claim 1, wherein the identifyingthe set of occluded pixels and the set of non-occluded pixels comprisesdetermining a difference between the estimated high resolution image andone of a depth map or a stereo disparity map, wherein the one of thedepth map or the stereo disparity map is based on a pixel shiftrepresentative of a depth of pixels at different spatial viewpoints. 4.The method of claim 1, wherein the plurality of low resolution imagescomprises images of a single subject from different spatial viewpoints.5. The method of claim 1, wherein the penalty function includes a MarkovRandom Field based on one of a depth map or a stereo disparity map. 6.The method of claim 1, further comprising storing the corrected highresolution image as a desired high resolution image in a memory inresponse to the value of the penalty function being below a thresholdvalue.
 7. A method comprising: determining a value of a penalty functionthat is based on a difference between a plurality of estimated lowresolution images and a plurality of low resolution images; determininga corrected high resolution image by making an adjustment to anestimated high resolution image, wherein the adjustment is made byreducing the difference using an iterated conditional mode; anddetermining a plurality of corrected low resolution images bydown-sampling the corrected high resolution image.
 8. The method ofclaim 5, wherein a desired high resolution image is obtained in responseto the value of the penalty function being below a threshold value. 9.The method of claim 5, wherein minimizing the disparity is based on aMarkov Random Fields.
 10. The method of claim 5, wherein an estimatedlow resolution image for a group G of high resolution pixels of theestimated high resolution image that averages from a high resolutionpixel (l₁,l₂) to a low resolution pixel (n₁,n₂) using a down-samplingoperator d(n₁,n₂,l₁,l₂) is estimated in accordance with therelationship: $\begin{matrix}{{y_{1}( {n_{1},n_{2}} )} = {{\sum\limits_{l_{1,}l_{2}ɛ\; G}{{d( {n_{1},n_{2},{l_{1,}l_{2}}} )} \cdot {x( {l_{1,}l_{2}} )}}} + {\eta_{1}( {n_{1},n_{2}} )}}} & \;\end{matrix}$
 11. The method of claim 10, wherein an estimated lowresolution image y₂(n₁,n₂) for the group G of high resolution pixels ofthe estimated high resolution image that averages from the highresolution pixel (l₁,l₂) with a disparity value δ(l₁) to the lowresolution pixel (n₁,n₂) using the down-sampling operator d(n₁,n₂,l₁,l₂)is estimated in accordance with the relationship:${y_{1}( {n_{1},n_{2}} )} = {{\sum\limits_{l_{1,}l_{2}ɛ\; G}{{d( {n_{1},n_{2},{l_{1,}l_{2}}} )} \cdot {x( {{l_{1}{\delta ( l_{1} )}},l_{2}} )}}} + {\eta_{2}( {n_{1},n_{2}} )}}$12. The method of claim 11, wherein the penalty function for theestimated low resolution images (y₁(n₁,n₂)) and (y₂(n₁,n₂)) is estimatedin accordance with the relationship:${E_{d}( {\delta,x} )} = {{\sum\limits_{n_{1},{n_{2} = 1}}^{N_{1},N_{2}}( {{y_{1}( {n_{1},n_{2}} )} - {\sum\limits_{l_{1},{l_{2} \in G}}{{d( {n_{1},n_{2},l_{1},l_{2}} )}{x( {l_{1},l_{2}} )}}}} )^{2}} + {\sum\limits_{n_{1},{n_{2} \in O^{\prime}}}( {{y_{2}( {n_{1},n_{2}} )} - {\sum\limits_{l_{1},{l_{2} \in G}}{{d( {n_{1},n_{2},l_{1},l_{2}} )}{x( {{\theta ( l_{1} )},l_{2}} )}}}} )^{2}} + {\sum\limits_{n_{1},{n_{2} \in O}}\lambda_{OCC}}}$wherein: Θ(l₁)=l₁−δ(l₁), 0 is the set of low resolution pixels each ofwhich involves contribution from a high resolution site for which thehigh resolution pixel is occluded, and 0′ is the compliment of
 0. 13.The method of claim 5, wherein an estimated low resolution imagecomprises a low resolution pixel having an intensity that is based on anaverage of one or more intensities of one or more high resolution pixelsof the estimated high resolution image, wherein the plurality ofestimated low resolution images includes the estimated low resolutionimage.
 14. The method of claim 13, wherein the intensity of theestimated low resolution pixel is further based on a second disparitybetween the low resolution pixel and the average of the one or more highresolution pixels.
 15. The method of claim 5, wherein the differencebetween the plurality of estimated low resolution images and theplurality of low resolution images is based on one or more comparisonsbetween the plurality of estimated low resolution images and a pluralityof down-sampled low resolution images, wherein the plurality ofdown-sampled low resolution images is based on the plurality of lowresolution images and the estimated high resolution Image.
 16. A systemcomprising: an image processing circuit configured to: generate acorrected high resolution image by iteratively: determining a pluralityof estimated low resolution images by down-sampling an estimated highresolution image; determining a value of a penalty function, wherein thepenalty function is based on a difference between the plurality ofestimated low resolution images and a plurality of low resolutionimages; and determining an adjustment of one or more pixels of theestimated high resolution image based on a second difference, whereinapplying the adjustment reduces the value of the penalty function 17.The system of claim 16, further comprising at least one camera to obtainthe plurality of low resolution images.
 18. The system of claim 17,wherein the plurality of low resolution images comprise stereo imagesobtained from one of a plurality of stereo cameras or a translation of astereo camera in different spatial viewpoints.
 19. The system of claim16, wherein the plurality of estimated low resolution images aregenerated by averaging one or more pixels of the estimated highresolution image.